34 research outputs found
Understanding Interventional TreeSHAP : How and Why it Works
Shapley values are ubiquitous in interpretable Machine Learning due to their
strong theoretical background and efficient implementation in the SHAP library.
Computing these values used to induce an exponential cost with respect to the
number of input features of an opaque model. Now, with efficient
implementations such as Interventional TreeSHAP, this exponential burden is
alleviated assuming one is explaining ensembles of decision trees. Although
Interventional TreeSHAP has risen in popularity, it still lacks a formal proof
of how/why it works. We provide such proof with the aim of not only increasing
the transparency of the algorithm but also to encourage further development of
these ideas. Notably, our proof for Interventional TreeSHAP is easily adapted
to Shapley-Taylor indices
Detection and Evaluation of bias-inducing Features in Machine learning
The cause-to-effect analysis can help us decompose all the likely causes of a
problem, such as an undesirable business situation or unintended harm to the
individual(s). This implies that we can identify how the problems are
inherited, rank the causes to help prioritize fixes, simplify a complex problem
and visualize them. In the context of machine learning (ML), one can use
cause-to-effect analysis to understand the reason for the biased behavior of
the system. For example, we can examine the root causes of biases by checking
each feature for a potential cause of bias in the model. To approach this, one
can apply small changes to a given feature or a pair of features in the data,
following some guidelines and observing how it impacts the decision made by the
model (i.e., model prediction). Therefore, we can use cause-to-effect analysis
to identify the potential bias-inducing features, even when these features are
originally are unknown. This is important since most current methods require a
pre-identification of sensitive features for bias assessment and can actually
miss other relevant bias-inducing features, which is why systematic
identification of such features is necessary. Moreover, it often occurs that to
achieve an equitable outcome, one has to take into account sensitive features
in the model decision. Therefore, it should be up to the domain experts to
decide based on their knowledge of the context of a decision whether bias
induced by specific features is acceptable or not. In this study, we propose an
approach for systematically identifying all bias-inducing features of a model
to help support the decision-making of domain experts. We evaluated our
technique using four well-known datasets to showcase how our contribution can
help spearhead the standard procedure when developing, testing, maintaining,
and deploying fair/equitable machine learning systems.Comment: 65 pages, manuscript accepted at EMSE journal, manuscript number,
EMSE-D-22-00330R
Microfluidic multipoles: theory and applications
Microfluidic multipoles (MFMs) have been realized experimentally and hold
promise for "open-space" biological and chemical surface processing. Whereas
convective flow can readily be predicted using hydraulic-electrical analogies,
the design of advanced MFMs is constrained by the lack of simple, accurate
models to predict mass transport within them. In this work, we introduce the
first exact solutions to mass transport in multipolar microfluidics based on
the iterative conformal mapping of 2D advection-diffusion around a simple edge
into dipoles and multipolar geometries, revealing a rich landscape of transport
modes. The models were validated experimentally with a library of 3D printed
MFM devices and found in excellent agreement. Following a theory-guided design
approach, we further ideated and fabricated two new classes of spatiotemporally
reconfigurable MFM devices that are used for processing surfaces with
time-varying reagent streams, and to realize a multistep automated immunoassay.
Overall, the results set the foundations for exploring, developing, and
applying open-space MFMs.Comment: 16 pages, 5 figure
Statistical Analysis of Spherical Harmonics Representations of Soil Particles
RÉSUMÉ :Grâce aux avancées en micro-tomographie par rayons-X, il est désormais possible d’obtenir des représentations en 3D haute résolution de milliers de particules échantillonnées depuis diverses sources géologiques. La représentation plus précise des particules pourrait éventuellement permettre d’obtenir des simulations numériques plus fidèles des comportements de matériaux granulaires par la méthode des éléments discrets (DEM, Discrete Element Method en anglais). Cependant, l’accès à des descriptions fines demande aussi de développer de nouveaux outils numériques pour la caractérisation géométrique et l’analyse statistique d’ensembles
de particules. Ce mémoire se concentre sur la modélisation géométrique des particules de sol par la représentation de leur surface à l’aide de la décomposition en harmoniques
sphériques. Plus précisément, nous discutons de l’utilisation des représentations en harmoniques
sphériques pour développer un modèle statistique permettant de générer des assemblages virtuels de particules à partir des données de plusieurs centaines de grains. La haute dimension de tels ensembles de données a longtemps été une complication majeure, mais avec les récentes avancées en apprentissage automatique dans l’analyse des mégadonnées,
il y a espoir que ces nouveaux algorithmes puissent surmonter cette limitation.----------ABSTRACT : Advancements in X-ray micro-computed tomography allow one to obtain high resolution 3D representations of particles collected from multiple geological sources. The representational power enabled by this new technology could allow for more accurate numerical simulations of granular materials using the celebrated Discrete Element Method (DEM). However, access to
realistic representations of particles requires the development of more advanced geometrical and statistical characterization techniques. This thesis focuses on the use of the Spherical Harmonics decomposition of soil particles to model the surface of the particles. More precisely,
we discuss the application of the Spherical Harmonics decomposition of particles to develop generative models of virtual assemblies that are calibrated based on datasets made of hundreds of grains. For long, the high dimensionality of the data has been a major challenge
to the developpement of such statistical models. However, with recent advances of machine learning algorithms in the context of Big Data, there is hope that these new techniques can be utilized to overcome this limitation and obtain very accurate generative models of assemblies
Recommended from our members
Corticocortical feedback increases the spatial extent of normalization
Normalization has been proposed as a canonical computation operating across different brain regions, sensory modalities, and species. It provides a good phenomenological description of non-linear response properties in primary visual cortex (V1), including the contrast response function and surround suppression. Despite its widespread application throughout the visual system, the underlying neural mechanisms remain largely unknown. We recently observed that corticocortical feedback contributes to surround suppression in V1, raising the possibility that feedback acts through normalization. To test this idea, we characterized area summation and contrast response properties in V1 with and without feedback from V2 and V3 in alert macaques and applied a standard normalization model to the data. Area summation properties were well explained by a form of divisive normalization, which computes the ratio between a neuron's driving input and the spatially integrated activity of a “normalization pool.” Feedback inactivation reduced surround suppression by shrinking the spatial extent of the normalization pool. This effect was independent of the gain modulation thought to mediate the influence of contrast on area summation, which remained intact during feedback inactivation. Contrast sensitivity within the receptive field center was also unaffected by feedback inactivation, providing further evidence that feedback participates in normalization independent of the circuit mechanisms involved in modulating contrast gain and saturation. These results suggest that corticocortical feedback contributes to surround suppression by increasing the visuotopic extent of normalization and, via this mechanism, feedback can play a critical role in contextual information processing
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
Context: Machine Learning (ML) has been at the heart of many innovations over
the past years. However, including it in so-called 'safety-critical' systems
such as automotive or aeronautic has proven to be very challenging, since the
shift in paradigm that ML brings completely changes traditional certification
approaches.
Objective: This paper aims to elucidate challenges related to the
certification of ML-based safety-critical systems, as well as the solutions
that are proposed in the literature to tackle them, answering the question 'How
to Certify Machine Learning Based Safety-critical Systems?'.
Method: We conduct a Systematic Literature Review (SLR) of research papers
published between 2015 to 2020, covering topics related to the certification of
ML systems. In total, we identified 217 papers covering topics considered to be
the main pillars of ML certification: Robustness, Uncertainty, Explainability,
Verification, Safe Reinforcement Learning, and Direct Certification. We
analyzed the main trends and problems of each sub-field and provided summaries
of the papers extracted.
Results: The SLR results highlighted the enthusiasm of the community for this
subject, as well as the lack of diversity in terms of datasets and type of
models. It also emphasized the need to further develop connections between
academia and industries to deepen the domain study. Finally, it also
illustrated the necessity to build connections between the above mention main
pillars that are for now mainly studied separately.
Conclusion: We highlighted current efforts deployed to enable the
certification of ML based software systems, and discuss some future research
directions.Comment: 60 pages (92 pages with references and complements), submitted to a
journal (Automated Software Engineering). Changes: Emphasizing difference
traditional software engineering / ML approach. Adding Related Works, Threats
to Validity and Complementary Materials. Adding a table listing papers
reference for each section/subsection
Mortality from gastrointestinal congenital anomalies at 264 hospitals in 74 low-income, middle-income, and high-income countries: a multicentre, international, prospective cohort study
Background: Congenital anomalies are the fifth leading cause of mortality in children younger than 5 years globally. Many gastrointestinal congenital anomalies are fatal without timely access to neonatal surgical care, but few studies have been done on these conditions in low-income and middle-income countries (LMICs). We compared outcomes of the seven most common gastrointestinal congenital anomalies in low-income, middle-income, and high-income countries globally, and identified factors associated with mortality. // Methods: We did a multicentre, international prospective cohort study of patients younger than 16 years, presenting to hospital for the first time with oesophageal atresia, congenital diaphragmatic hernia, intestinal atresia, gastroschisis, exomphalos, anorectal malformation, and Hirschsprung's disease. Recruitment was of consecutive patients for a minimum of 1 month between October, 2018, and April, 2019. We collected data on patient demographics, clinical status, interventions, and outcomes using the REDCap platform. Patients were followed up for 30 days after primary intervention, or 30 days after admission if they did not receive an intervention. The primary outcome was all-cause, in-hospital mortality for all conditions combined and each condition individually, stratified by country income status. We did a complete case analysis. // Findings: We included 3849 patients with 3975 study conditions (560 with oesophageal atresia, 448 with congenital diaphragmatic hernia, 681 with intestinal atresia, 453 with gastroschisis, 325 with exomphalos, 991 with anorectal malformation, and 517 with Hirschsprung's disease) from 264 hospitals (89 in high-income countries, 166 in middle-income countries, and nine in low-income countries) in 74 countries. Of the 3849 patients, 2231 (58·0%) were male. Median gestational age at birth was 38 weeks (IQR 36–39) and median bodyweight at presentation was 2·8 kg (2·3–3·3). Mortality among all patients was 37 (39·8%) of 93 in low-income countries, 583 (20·4%) of 2860 in middle-income countries, and 50 (5·6%) of 896 in high-income countries (p<0·0001 between all country income groups). Gastroschisis had the greatest difference in mortality between country income strata (nine [90·0%] of ten in low-income countries, 97 [31·9%] of 304 in middle-income countries, and two [1·4%] of 139 in high-income countries; p≤0·0001 between all country income groups). Factors significantly associated with higher mortality for all patients combined included country income status (low-income vs high-income countries, risk ratio 2·78 [95% CI 1·88–4·11], p<0·0001; middle-income vs high-income countries, 2·11 [1·59–2·79], p<0·0001), sepsis at presentation (1·20 [1·04–1·40], p=0·016), higher American Society of Anesthesiologists (ASA) score at primary intervention (ASA 4–5 vs ASA 1–2, 1·82 [1·40–2·35], p<0·0001; ASA 3 vs ASA 1–2, 1·58, [1·30–1·92], p<0·0001]), surgical safety checklist not used (1·39 [1·02–1·90], p=0·035), and ventilation or parenteral nutrition unavailable when needed (ventilation 1·96, [1·41–2·71], p=0·0001; parenteral nutrition 1·35, [1·05–1·74], p=0·018). Administration of parenteral nutrition (0·61, [0·47–0·79], p=0·0002) and use of a peripherally inserted central catheter (0·65 [0·50–0·86], p=0·0024) or percutaneous central line (0·69 [0·48–1·00], p=0·049) were associated with lower mortality. // Interpretation: Unacceptable differences in mortality exist for gastrointestinal congenital anomalies between low-income, middle-income, and high-income countries. Improving access to quality neonatal surgical care in LMICs will be vital to achieve Sustainable Development Goal 3.2 of ending preventable deaths in neonates and children younger than 5 years by 2030
Mortality from gastrointestinal congenital anomalies at 264 hospitals in 74 low-income, middle-income, and high-income countries: a multicentre, international, prospective cohort study
Summary
Background Congenital anomalies are the fifth leading cause of mortality in children younger than 5 years globally.
Many gastrointestinal congenital anomalies are fatal without timely access to neonatal surgical care, but few studies
have been done on these conditions in low-income and middle-income countries (LMICs). We compared outcomes of
the seven most common gastrointestinal congenital anomalies in low-income, middle-income, and high-income
countries globally, and identified factors associated with mortality.
Methods We did a multicentre, international prospective cohort study of patients younger than 16 years, presenting to
hospital for the first time with oesophageal atresia, congenital diaphragmatic hernia, intestinal atresia, gastroschisis,
exomphalos, anorectal malformation, and Hirschsprung’s disease. Recruitment was of consecutive patients for a
minimum of 1 month between October, 2018, and April, 2019. We collected data on patient demographics, clinical
status, interventions, and outcomes using the REDCap platform. Patients were followed up for 30 days after primary
intervention, or 30 days after admission if they did not receive an intervention. The primary outcome was all-cause,
in-hospital mortality for all conditions combined and each condition individually, stratified by country income status.
We did a complete case analysis.
Findings We included 3849 patients with 3975 study conditions (560 with oesophageal atresia, 448 with congenital
diaphragmatic hernia, 681 with intestinal atresia, 453 with gastroschisis, 325 with exomphalos, 991 with anorectal
malformation, and 517 with Hirschsprung’s disease) from 264 hospitals (89 in high-income countries, 166 in middleincome
countries, and nine in low-income countries) in 74 countries. Of the 3849 patients, 2231 (58·0%) were male.
Median gestational age at birth was 38 weeks (IQR 36–39) and median bodyweight at presentation was 2·8 kg (2·3–3·3).
Mortality among all patients was 37 (39·8%) of 93 in low-income countries, 583 (20·4%) of 2860 in middle-income
countries, and 50 (5·6%) of 896 in high-income countries (p<0·0001 between all country income groups).
Gastroschisis had the greatest difference in mortality between country income strata (nine [90·0%] of ten in lowincome
countries, 97 [31·9%] of 304 in middle-income countries, and two [1·4%] of 139 in high-income countries;
p≤0·0001 between all country income groups). Factors significantly associated with higher mortality for all patients
combined included country income status (low-income vs high-income countries, risk ratio 2·78 [95% CI 1·88–4·11],
p<0·0001; middle-income vs high-income countries, 2·11 [1·59–2·79], p<0·0001), sepsis at presentation (1·20
[1·04–1·40], p=0·016), higher American Society of Anesthesiologists (ASA) score at primary intervention
(ASA 4–5 vs ASA 1–2, 1·82 [1·40–2·35], p<0·0001; ASA 3 vs ASA 1–2, 1·58, [1·30–1·92], p<0·0001]), surgical safety
checklist not used (1·39 [1·02–1·90], p=0·035), and ventilation or parenteral nutrition unavailable when needed
(ventilation 1·96, [1·41–2·71], p=0·0001; parenteral nutrition 1·35, [1·05–1·74], p=0·018). Administration of
parenteral nutrition (0·61, [0·47–0·79], p=0·0002) and use of a peripherally inserted central catheter (0·65
[0·50–0·86], p=0·0024) or percutaneous central line (0·69 [0·48–1·00], p=0·049) were associated with lower mortality.
Interpretation Unacceptable differences in mortality exist for gastrointestinal congenital anomalies between lowincome,
middle-income, and high-income countries. Improving access to quality neonatal surgical care in LMICs will
be vital to achieve Sustainable Development Goal 3.2 of ending preventable deaths in neonates and children younger
than 5 years by 2030
Learning Hybrid Interpretable Models: Theory, Taxonomy, and Methods
A hybrid model involves the cooperation of an interpretable model and a complex black box. At inference, any input of the hybrid model is assigned to either its interpretable or complex component based on a gating mechanism. The advantages of such models over classical ones are two-fold: 1) They grant users precise control over the level of transparency of the system and 2) They can potentially perform better than a standalone black box since redirecting some of the inputs to an interpretable model implicitly acts as regularization. Still, despite their high potential, hybrid models remain under-studied in the interpretability/explainability literature. In this paper, we remedy this fact by presenting a thorough investigation of such models from three perspectives: Theory, Taxonomy, and Methods. First, we explore the theory behind the generalization of hybrid models from the Probably-Approximately-Correct (PAC) perspective. A consequence of our PAC guarantee is the existence of a sweet spot for the optimal transparency of the system. When such a sweet spot is attained, a hybrid model can potentially perform better than a standalone black box. Secondly, we provide a general taxonomy for the different ways of training hybrid models: the Post-Black-Box and Pre-Black-Box paradigms. These approaches differ in the order in which the interpretable and complex components are trained. We show where the state-of-the-art hybrid models Hybrid-Rule-Set and Companion-Rule-List fall in this taxonomy. Thirdly, we implement the two paradigms in a single method: HybridCORELS, which extends the CORELS algorithm to hybrid modeling. By leveraging CORELS, HybridCORELS provides a certificate of optimality of its interpretable component and precise control over transparency. We finally show empirically that HybridCORELS is competitive with existing hybrid models, and performs just as well as a standalone black box (or even better) while being partly transparent
Fooling SHAP with Stealthily Biased Sampling
SHAP explanations aim at identifying which features contribute the most to
the difference in model prediction at a specific input versus a background
distribution. Recent studies have shown that they can be manipulated by
malicious adversaries to produce arbitrary desired explanations. However,
existing attacks focus solely on altering the black-box model itself. In this
paper, we propose a complementary family of attacks that leave the model intact
and manipulate SHAP explanations using stealthily biased sampling of the data
points used to approximate expectations w.r.t the background distribution. In
the context of fairness audit, we show that our attack can reduce the
importance of a sensitive feature when explaining the difference in outcomes
between groups, while remaining undetected. These results highlight the
manipulability of SHAP explanations and encourage auditors to treat post-hoc
explanations with skepticism